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SUMMARY  OF  FINDINGS 


A.  ) INTRODUCTION 

~ir  Teleradiology  Is  an  automated  system  whereby  an  electronic 
representation  of  an  X-ray  Image  is  transmitted  via  telephone  wires 
from  one  location  to  another.  \  The  TRIMIS  Program  Office  (Department 
of  Defense)  and  the  Bureau  olj  Radiological  Health  (Public  Health 
Service)  have*  conducted  a  six-month  field  trial  of  a  Teleradiology 
system.  The  system,  which  was  ^signed  by  the  MITRE  Corporation,  was 
studied  to  evaluate  its  performance  in  a  routine  medical  practice 
setting.  "  /  !<'/+"  I  *  /  rj 

In  the  fall  of  1980,  four  small  clinics  in  the  Washington,  D.C. 
area  were  chosen  as  transmitter  sites  for  the  trial  and  a  large 
radiology  department  was  selected  as  the  central  receiver  site. 

Training  for  site  personnel  began  in  the  fall  of  1981.  The  system  was 
Installed  in  January  of  1982,  becoming  fully  operational  In  late 
March,  and  remaining  so  through  June.  Baseline  (pre-implementation) 
data  collection  for  the  functional  evaluation  of  the  Teleradiology 
system  was  conducted  by  the  Bureau  of  Radiological  Health  for  two 
weeks  in  February  of  1981  and  by  Arthur  D.  Little,  Inc.  for  two  weeks 
in  September  of  that  year.  Post-implementation  data  were  collected  by 
Arthur  D.  Little,  Inc.  for  two  weeks  in  June  of  1982. 

The  Teleradiology  system  was  experimental  and,  hence,  was  not 
used  as  a  replacement  for  normal  film  Interpretation  of  X-ray  exams 
during  the  field  trial.  The  "manual"  systems  used  by  each  of  the 
transmitter  clinics  continued  to  be  used  in  parallel  with 
Teleradiology.  Because  of  the  parallel  use  of  Teleradiology  and  yor 
"manual"  methods  for  obtaining  Interpretations,  many  of  the  data  *1 
collected  for  the  functional  evaluation  were  designed  to  estimate  the 

system's  potential  Impact,  rather  than  to  directly  measure  its  Impact  tion - 

in  the  experimental  setting. 

By - 

Data  were  collected  concerning:  Distribution/ 

e  the  potential  Impact  of  the  system  on  patient  care;  Availability  Codes 

•  the  acceptability  of  the  system  to  users;  i. \  Avail  and/or 

/  \  Dist  Special 
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•  the  potential  feasibility  of  using  the  system  in  routine 
practice  settings;  and 

e  the  potential  Impact  of  the  system  on  coats  of  transmitter 
site  X-ray  services. 

B.  THE  STUDY  SETTING 

The  central  receiver  site  for  the  Teleradiology  field  trial  was 
the  Radiology  Department  at  Malcolm  Grow  Medical  Center  at  Andrews  Air 
Force  Base  in  Maryland.  During  the  field  trial*  this  large  radiology 
department  continued,  for  the  most  part,  to  operate  as  it  had  before. 
The  Teleradiology  equipment  was  physically  separated  from  the  other 
viewing  rooms  in  the  department,  and  both  the  radiologists  and  the 
tran8crlptloni8ts  who  used  the  system  were  hired  for  the  project. 

The  four  medical  facilities  used  as  transmitter  sites  during  the 
field  trial  were: 

e  Bolling  Air  Force  Base  Clinic,  Washington,  D.C.; 

e  Fort  Detrlck  Army  Clinic,  Frederick,  Maryland; 

e  Patuxent  Naval  Air  Station  Hospital,  Lexington  Park, 

Maryland;  and 

e  Central  Virginia  Community  Health  Center,  New  Canton, 

Virginia. 

These  clinics  and  their  X-ray  departments  are  quite  small  and  are 
oriented  primarily  toward  the  delivery  of  outpatient  services.  The 
clinics  vary  in  workload  from  21,000  outpatient  visits  per  year  to 
85,000.  Their  X-ray  departments  range  from  having  one  technician,  one 
X-ray  room  and  examining  1,400  patients  per  year  to  four  technicians, 
two  rooms  and  7,000  patients  per  year.  Each  of  the  four  sites  has 
standard  arrangements  with  specific  medical  centers  for  secondary  and 
tertiary  care  referrals.  Their  distance  to  these  medical  centers 
varies  from  10  to  90  miles.  Normally,  two  of  the  four  clinics  send 
their  X-ray  films  by  courier  to  medical  centers  for  interpretation; 
one  employs  a  part-time  radiologist  to  perform  film  Interpretations; 
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and  the  fourth  sands  films  away  avary  othar  weak*  employing  a  visiting 
radiologist  during  tha  off-vaaks. 

During  tha  fiald  trial*  tha  transmitter  departments  continued  to 
have  their  films  read  "manually*"  as  they  had  before.  At  each  of  four 
of  the  transmitter  sites*  the  regular  technicians  were  trained  to  use 
the  Teleradiology  system  and  did  operate  it.  However*  at  three  of  the 
sites*  additional  temporary  personnel  were  also  hired  to  accommodate 
system  use. 

C.  RESULTS;  REAL  AMD  POTENTIAL  IMPACTS 

1.  Impact  on  Patient  Care 

The  potential  impact  of  Teleradiology  on  patient  care  was  studied 
in  two  ways.  First*  turnaround  time  of  Interpretation  reports  was 
measured  for  all  exams  ordered  during  the  two  study  periods  (before 
and  after  the  Teleradiology  system  was  Installed).  And  second*  a 
study  was  conducted  of  how  radiographs  and  X-ray  interpretations  were 
used  in  patient  care  at  the  transmitter  facilities.  All  data 
regarding  turnaround  time  and  patient  care  impacts  were  collected 
using  a  self-administered  survey  form,  which  followed  each  X-ray 
through  the  various  stages  of  the  X-ray  request /report  cycle  and  was 
completed  by  clinic  staff.  Data  were  collected  regarding  a  total  of 
418  patients  (453  exams)  before  system  Implementation  and  for  618 
patients  (695  exams)  during  system  operation. 
a.  Turnaround  Time:  X-Ray  Request /Report  Cycle 

Over  90Z  of  the  X-rays  at  each  of  the  four  clinics  and  during 
both  data  collection  periods  were  performed  within  two  hours  of  being 
ordered.  However*  delays  of  severe*,  days  often  occurred  between  the 
exam' 8  performance  and  its  interpretation  by  a  radiologist  and  between 
this  interpretation  and  its  review  by  the  referring  provider. 

The  total  mean  turnaround  time  required  for  "manual”  film 
interpretation  reports  before  system  implementation  was  found  to  range 
from  88  hours  to  108  hours  at  the  four  study  sites.  During  system 
operation*  mean  turnaround  time  for  "manual"  film  Interpretation 
reports  was  somewhat  longer*  ranging*  at  the  the  four  sites,  from  121 
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to  233  hours.  This  increase  In  turnaround  tins  nay  bs  associated 
with: 

s  the  parallel  use  of  the  two  interpretation  sodalities 

during  the  post-lap lementat ion  data  collection  period;  and 

s  miscellaneous  breakdowns  in  the  "manual"  interpretation 
systems  during  the  post-implementation  data  collection 
period. 

The  total  mean  turnaround  time  required  for  automated 
Teleradiology  interpretations  was  even  longer  than  that  for  manual 
services.  This  mean  varied  from  181  to  212  hours  at  the  four  sites. 

Delays  for  receipt  of  Teleradiology  interpretations  were  largely 
attributable  to  the  fact  that  the  system  was  only  used  experimentally: 

e  the  system  did  not  initially  function  reliably; 

e  X-ray  images  were  not  always  input  regularly  or  on  a  dally 
basis ; 

e  radiologists  were  only  available  to  perform  Teleradiology 
interpretations  during  morning  hours  five  days  each  week; 

e  "manual"  interpretation  reports  were  often  received  prior 
to  Teleradiology  Interpretations  of  the  same  exam,  and, 
hence,  were  not  always  read  promptly. 

In  order  to  project  the  impact  of  a  refined  Teleradiology  system 
in  routine  use  (a  non-experlaental  setting),  it  is  assumed  that  the 
system  would  be  implemented  and  utilized  quite  differently.  For 
example: 

e  a  non-experimental  system  would  probably  be  more  reliable; 

e  if  the  system  were  used  routinely,  a  protocol  would 
probably  be  established  for  inputting  films  into  the 
system  regularly; 

e  a  radiologist  would  probably  be  available  on  a  full-time 
basis;  and 

•  no  parallel  "manual"  system  would  be  available. 

Based  on  these  assumptions,  the  data  collected  during  the  post- 
implementation  period  suggest  that  in  a  non-experimental  situation, 
turnaround  time  for  Teleradiology  interpretation  reports  could  be  much 
ehorter  than  that  observed  during  the  field  trial.  Inputting  of  films 
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requires  approximately  10  minutes  per  exam*  and  telephone  line 
transmission  takes  an  additional  15  to  30  minutss  per  exam.  As  soon 
as  transmission  Is  complste*  Interpretation  may  be  performed*  and 
either  the  telephone  or  the  tele-typewriter  nay  he  used  to 
communicate  findings  immediately  to  remote  providers.  We  project  that 
a  24-hour  total  turnaround  would  be  possible  for  routine  exams  and 
that  a  1-hour  turnaround  could  be  accomplished  for  STAT  exams . 

Impact  on  Patient  Care 


Data  were  collected  concerning  the  types  of  X-rays  performed 
during  the  two  study  periods  at  the  transmitter  site  X-ray 
departments.  Also,  for  each  X-ray  ordered  during  the  study  periods, 
referring  providers  were  asked  to  answer  three  different  questions 
concerning  the  use  of  Interpretations  in  patient  care.  These 
questions  concerned  the  relative  significance  of  prompt  receipt  of  a 
radiologist's  Interpretation;  the  role  of  X-ray  film  viewing  in 
patient  care  decisions;  and  the  role  of  the  radiologist's  report  in 
these  decisions.  Little  variation  existed  between  the  two  data 
collection  periods  in  the  types  of  exams  performed  and  the  patterns  of 
provider  response  to  "patient  care"  questions.  Hence*  these  data  have 
been  pooled  for  presentation. 

The  X-ray  exams  performed  at  the  study  sites  were  grouped  into 
four  categories  corresponding  to  various  clinical  uses: 
e  Routine  physical  chest  exams; 
e  Emergency  exams  associated  with  acute  trauma; 
e  Diagnostic  exams  not  associated  with  acute  trauma;  and 
e  X-rays  taken  "for  the  record"  or  as  follow-up  exams. 

The  case  mix  in  the  X-ray  departments  at  each  of  the  transmitter 
facilities  is  quite  limited*  as  would  be  expected  in  primary  care 
clinics.  Also,  the  distribution  of  X-rays  performed  at  the  military 
sites  is  somewhat  different  from  that  at  the  civilian  clinic.  Central 
Virginia  Community  Health  Center  (CVCHC) .  During  the  two  study 
periods,  23Z  of  the  X-rays  performed  at  the  military  sites  were 
associated  with  routine  physicals  (compared  with  6%  at  CVCHC) ;  40Z 
were  associated  with  acute  trauma  (compared  with  28%  at  CVCHC); 
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17Z  were  other  "diagnostic"  mu  (coopered  with  50Z  at  CVCHC) ;  and 
20Z  vara  performed  "for  the  record"  or  aa  follow-up  procaduraa  (versus 
16Z  at  CVCHC) . 

At  the  tine  that  the  exan  was  requested*  providers  ware  naked  to 
categorise  how  significant  proapt  receipt  of  a  radiologist's 
Interpretation  would  be  In  patient  care.  In  73Z  of  cases*  providers 
Indicated  that  prompt  receipt  of  a  radiologist's  Interpretation  would 
have  some  effect  on  their  opinions  or  decisions  regarding  patient 
care.  Timely  Interpretation  receipt  was  considered  "very  significant: 
essential  to  patient  care  decisions"  In  only  8Z  of  cases.  Rapid 
interpretation  turnaround  was  felt  to  be  most  Important  for  exams 
associated  with  trauma  and  other  diagnostic  exams  and  least 
significant  for  routine  physicals,, 

After  X-rays  were  performed  at  the  transmitter  facilities,  the 
requesting  providers-  viewed  the  films  themselves  In  a  majority  of 
cases.  Providers  were  most  likely  to  review  exams  associated  with 
acute  trauma  (77Z),  followed  by  non-emergency  exams  that  they 
considered  diagnostic  (69Z)  and  radiographs  performed  "for  the  record" 
or  as  follow-up  procedures  (62Z).  They  seldom  viewed  routine  physical 
chest  exams  (10Z). 

If  and  when  the  referring  provider  viewed  the  films  that  he  had 
ordered,  he  was  asked  to  categorize  how  he  felt  this  viewing  had 
affected  his  handling  of  the  case.  Usually  (In  63Z  of  cases)  the 
viewing  of  radiographs  served  to  "increase  the  clinical  confidence"  of 
providers.  Sometimes  —  primarily  In  trauma  cases  or  for  other 
diagnostic  exams  —  providers  reported  that  the  film  viewing  had  had  a 
major  effect  on  handling  of  the  case  (16Z  and  21Z  of  cases* 
respectively).  It  was  least  likely  to  have  had  a  major  effect  for 
routine  physicals  (4Z  of  cases). 

At  the  time  the  provider  reviewed  the  radiologist's  film  or 
Teleradiology  interpretation  report*  he  was  asked  what  effect  the 
specialist's  report  had  made  on  patient  treatment/disposition 
decisions.  At  the  transmitter  sites  radiologists'  interpretations 
were  almost  always  received  several  days  after  the  X-ray  had  been 
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performed  and  the  patient  treated  and  aent  hone  (see  above).  Reports 
received  after  bo  long  a  delay  would  not  be  expected  to  have  much 
effect  on  patient  care  unless  their  findings  differed  substantially 
from  those  aade  earlier  by  the  referring  provider.  Indeed*  by  the 
tine  radiologists'  reports  were  reviewed*  they  were  felt  to  have  had 
no  effect  no  patient  care  for  41Z  of  cases.  Forty-three  percent  of 
radiologists'  reports  were  felt  to  have  "Increased  the  clinical 
confidence"  of  the  providers.  It  Is  Interesting  to  note*  however* 
that  In  each  X-ray  category*  some  radiologists'  reports  were  reported 
to  have  had  a  major  effect  on  care  (6Z  for  routine  physicals;  8Z  for 
emergency  exams;  10Z  for  other  diagnostic  exams;  and  7Z  for  exams 
performed  "for  the  record"  or  as  follow-up  procedures) . 

2.  User  Acceptance 

During  the  post-implementation  data  collection  period*  system 
users  were  questioned  regarding  their  opinions  of  the  system  and  its 
utility.  The  response  rate  for  each  survey  was  over  90Z. 
a.  Acceptance  by  Primary  Care  Providers 

Overall,  providers*  comments  were  positive.  Most  of  their 
enthusiasm,  however*  was  derived  from  the  system's  potential  utility 
rather  than  actual  benefits  realized  during  the  field  trial. 
Providers  believed  that  the  system  could  be  valuable  where  turnaround 
time  reduced  to  a  few  hours  or  to  a  single  day.  Also*  at  some  of  the 
sites,  providers  felt  that  24-hour  availability  of  Interpretation 
services  was  important. 

.  Acceptance  by  Receiving  Site  Radiologists 


The  radiologists  who  had  participated  in  the  field  trial  at 
Malcolm  Grow  Medical  Center  completed  a  written  questionnaire 
regarding  their  experience  with  the  system.  The  radiologists' 
comments  were  enthusiastic.  They  felt  that  the  quality  of  images 
received  was  generally  good  and  that  image  resolution  was  usually 
adequate. 
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c.  Acceptance  by  Technicians 

Bach  technician  who  used  the  system  st  the  transmitter  sites  was 
interviewed.  All  felt  that  once  the  system  had  become  reliable*  it 
had  been  easy  to  use.  This  opinion  was  expressed  both  by  trained 
radiology  technicians  and  by  the  non-technician  system  operators.  The 
non-technicians  did  require  somewhat  more  time  to  become  accustomed  to 
the  system  —  to  learn  tha  correct  positioning  and  focusing  of  films 
—  but  soon  became  very  adept  at  its  operation. 

Transmitter  site  personnel  did  criticize  some  aspects  of  system 
design*  primarily  complaining  that  the  film  inputting  activity  was 
tedious  and  time-consuming. 

3.  Feasibility  of  Routine  Use 

In  order  to  determine  the  feasibility  of  using  the  Teleradiology 
system  in  routine  medical  practice  settings*  its  potential  Impact  on 
the  daily  routines  was  studied  at  both  the  transmitter  sites  and  at 
the  central  receiver  site. 

Several  days  of  work  sampling  were  performed  at  each  of  the  four 
transmitter  sites  both  before  and  after  system  Implementation.  During 
both  study  periods,  approximately  fifty  percent  of  X-ray  department 
staff  time  was  spent  in  activities  unrelated  to  X-ray  department  work. 
(Workload  is  extremely  uneven  in  these  small  departments,  and  much  of 
this  time  was  spent  "on-call"  for  X-ray  duty  during  non-busy  times.) 
Performing  and  processing  each  X-ray  and  doing  the  paperwork  and 
filing  associated  with  radiographs  took  between  20  minutes  and  40 
minutes  per  exam;  inputting  Teleradiology  images  required  an 
additional  10  minutes  per  exam.  It  is  assumed  that  these  figures  are 
good  estimates  of  the  amount  of  time  that  would  be  required  were  the 
system  in  routine  use.  Although  all  but  one  of  the  transmitter  sites 
did  Increase  their  staff  between  our  first  and  second  study  periods, 
the  data  fall  to  show  that  these  staff  Increases  would  be  required  to 
accommodate  routine  system  operation. 

Time  studies  performed  on  radiologists  interpreting  X-ray  films 
were  conducted  before  system  implementation  and  similar  studies  were 
conducted  under  Tele radiology.  It  was  determined  that  video  viewing 
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Is  no  nore  tine  consuming  than  flln  viewing,  end  elnllarly,  that 
typing  of  Teleradiology  interpretations  requires  no  nore  time  than 
regular  report  typing. 

From  these  work  sampling  and  tine  study  data,  one  could  infer 
that  snail  moderately  busy  X-ray  departments  should  be  able  to 
accommodate  Teleradiology  system  operation  into  their  daily  schedules 
without  an  increase  in  staff.  Large  radiology  departments  who 
currently  accommodate  interpretations  of  X-ray  films,  could  similarly 
accommodate  interpretations  of  the  Teleradiology  images  of  the  films. 
4.  System  Costs 

To  determine  the  potential  impact  of  Teleradiology  on  X-ray 
department  costs,  the  estimated  costs  of  the  Teleradiology  system  at 
the  four  transmitter  clinics  were  compared  with  the  two  "manual” 
methods  for  obtaining  radiologists'  interpretations  of  X-rays 
performed  at  these  clinics.  These  two  "manual"  methods  are  (1)  using 
a  courier  to  transport  films  and  (2)  employing  a  part-time  visiting 
radiologist.  In  the  field  trial  transmitter  sites,  the  equipment  and 
staff  necessary  to  perform  and  process  X-ray  exams  are  essentially  the 
same  regardless  of  which  of  the  three  systems  is  used.  Hence 
attention  was  focused  only  on  the  incremental  costs  associated  with 
the  three  alternatives.  The  cost  of  using  the  experimental 
Teleradiology  system  in  the  field  trial  sites  was  found  to  be 
approximately  $7  per  X-ray  exam,  compared  with  an  estimated  $2.50  per 
exam  irtten  a  part-time  visiting  radiologist  is  employed  and  $0.50  when 
a  courier  system  is  used.  These  relative  costs  would,  of  course,  vary 
in  different  settings. 

C.  SUMMARY  AND  CONCLUSIONS 

A  refined  Teleradiology  system  would  make  available  to  remote 
clinics  the  same  access  to  radiologists'  interpretation  services  as  is 
currently  available  to  large  hospital  outpatient  departments.  The 
reacts  clinics  would  have  routine  interpretations  returned  to 
providers  within  a  day  or  two  and  "wet  readings"  of  films  would  be 
accessible  at  any  time  during  the  day.  Primary  responsibility  for 
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radiological  interpretation  would  be  shifted  from  the  primary  care 
provider  to  the  radiologist. 

At  clinics  typified  by  the  four  transmitter  sites  Involved  in  the 
field  trial,  it  appears  that  Teleradiology  can  provide  a  much  more 
rapid  turnaround  of  radiologists'  Interpretation  reports  than  can 
either  of  the  standard  "manual"  methods  for  obtaining  film 
interpretations.  The  system  appears  feasible  to  use  in  small 
transmitter  site  X-ray  departments  and  a  large  central  receiving  site. 
It  has  been  demonstrated  to  be  acceptable  to  users.  At  the  field 
trial  sites,  alternative  methods  for  obtaining  interpretations  were 
readily  available,  and  were  less  expensive  than  using  Teleradiology. 

In  the  vast  majority  of  cases,  providers  in  transmitter  clinics 
expressed  a  preference  for  receiving  expert  Interpretations  promptly 
rather  than  relying  on  their  own  readings  of  films.  However,  the 
Importance  of  X-ray  examinations  and  the  significance  of  prompt 
receipt  of  radiologists'  interpretations  was  found  to  vary  with  X-ray 
exam  type.  Providers  Indicated  that  radiographic  findings  were  most 
relevant  to  Immediate  care  in  trauma  cases,  less  so  for  other 
diagnostic  exams,  followed  by  examinations  performed  "for  the  record" 
or  for  follow-up,  and,  finally,  they  were  felt  to  be  least  Immediately 
relevant  for  routine  physical  examinations. 
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